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CHAPTER 1 



INTRODUCTION 



PURPOSE 

The purpose of this research was to investigate the relationship between Design 
Input Index, as developed by James A. Broaddus in his dissertation, Desien 
Effectiveness in Constnicrion: The Relationship Between Inputs to the Design Process 
and Project Success , and change behavior of military construction projects. Change 
behavior included investigation of total changes, the cost of changes, change categories 
which represent design errors and omissions, and unforeseen changes. The goal of 
this work was to develop a statistically-based mathematical model to predict change 
behavior on large Navy projects if the Design Input Index was known. Having a 
model would allow executives and managers within the Engineering Field Division and 
the Resident Officer in Charge of Construction to predict the number and category of 
changes which might occur on a given project. This tool would be very powerful in 
decisions regarding allocation of the limited resources within the organization. 

SCOPE 

This research studied 55 projects within the Southeast United States. These 
were the same projects that Broaddus used in his dissertation. The Design Input Index 
numbers he developed were compared with project change information provided by 
Southern Division, Naval Facilities Engineering Command. The only change 
categories considered in the analysis were those that indicate an error or omission in the 
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design, and unforeseen conditions. The total number of changes and their associated 
costs were also included. These categories were chosen from the many available 
because they tie directly into the idea that a more well defined project will have fewer 
changes required and fewer mistakes. The analysis centered on finding a suitable 
statistical model for each category that could be used to predict the value based on a 
known Design Input Index. 



CHAPTER 2 



THE NAVY AS AN OWNER 



ORGANIZATION 

Like all large corporations, the United States Navy has a large number of 
facilities that must be maintained for the operating forces to be able to carry out their 
varied missions. The Navy must have facilities for submarines, surface vessels, and 
aircraft. There are also requirements to provide and maintain facilities for logistics, 
communications and personnel support facilities such as commissaries, exchanges, 
recreation facilities, etc. To meet these requirements in a changing world, the Navy has 
a capital improvement program called the Military Construction Program (MILCON). 
This program typically involves approximately $2 billion per year. This program 
replaces old and inefficient facilities and provides facilities needed because of new or 
revised missions for the operating forces.^ The Naval Facilities Engineering 
Command (NAVFAC) is the Navy's organization that administers and controls the 
MILCON Program. Figure 2-1 shows how NAVFAC fits in to the overall Department 
of Defense (DoD) and Navy organizations. NAVFAC is geographically organized into 
Engineering Field Divisions (EFD). These field divisions handle the actual 
accomplishment of major projects from conceptual planning through the final turnover 
to the using activity. Figure 2-2 is the Southern Division of NAVFAC Organization 
Chart. 



^ James A. Broaddus, Design Effectiveness in Constmction: The Relaiinnship Between Inniils 
to the Design Process and Project Success . Unpublished Dissertation, University of Texas at Austin, 
May 1991, p. 30. 
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FIGURE 2-1: NAVFAC IN THE DEPARTMENT OF DEFENSE 

ORGANIZATION 



All data used in this research came from the Southern Division of NAVFAC 
(SOUTHDIV). This EFD covers the geographic region in the Southern U. S. from 
South Carolina to New Mexico. It includes the states of South Carolina, Georgia, 
Tennessee, Florida, Alabama, Mississippi, Arkansas, Louisiana, Texas, Oklahoma, 
and New Mexico. There are 30 significant Navy shore installations, six Air Force 
bases and one Army activity within the area of responsibility. Most of the 1,000 people 
employed by SOUTHDIV are involved in conceptual planning, design, project 
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management, and construction activities. Most of those employees are based at the 
division's headquarters in Charleston, South Carolina. 

The EFD organization is divided further into field offices that handle the 
administration of the contract for a project once it has been awarded. These offices are 
the Resident Officer in Charge of Construction (ROICC). Within the ROICC office, 
there is a project team that handles contract administration. This team consists of a 
Project Manager, who is either a Navy Civil Engineer Corps Officer or a GS-1 1/12 
Civil Service engineer; a Construction Representative, who inspects the on-site work 
for conformance to project plans and specifications; and a Contract Specialist, who 
assists in handling the contractual matters associated with the project. 




FIGURE 2-2: SOUTHERN DIVISION ORGANIZATION CHART 
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CONSTRUCTION PROCESS 

The MILCON process begins years before any actual construction work is done 
on site. It starts with an activity having a requirement for a facility. This requirement 
can be generated locally at the affected base, or it may be generated by the needs of a 
new weapons system or a change in mission. After the requirement is identified, a 
project is submitted through the operational chain-of-command to the Chief of Naval 
Operations for validation. If the project is validated and is of high enough priority, it 
will become a part of the Six-Year Defense Program (SYDP). Not all Navy 
construction projects are a part of the MILCON process. In some instances, operations 
and maintenance funds may be used for construction, but the most complex and 
expensive projects performed are a part of the MILCON program. Each year, the 
Congress approves and funds each Navy new-construction project over $200,000 as an 
individual line-item of the Department of Defense's overall budget. This process is 
very competitive given the limited amount of funds available each year for overall 
defense spending. 

When a project is within three years of its budget year, the planning process 
begins in earnest. Scope is defined further so that there is sufficient information 
available to progress with design authorization. In this stage, the EFD Planning 
Department has control of the project. Once the project is "Certified Ready for Design" 
and the project is within two years of its budget year, design of the project can begin. 

The selection of the project A/E is based on qualifications per the Brooks Act, 
which is discussed later in this chapter. After the contract is negotiated, the A/E can 



7 



begin work on the detailed design. This period of the project is critical. The A/E must 
have the 35 percent design completed by the September that is 14 months before the 
project’s scheduled budget year. If they do not meet this milestone, the project will 
either be pushed back two years or it may be cancelled in its entirety. This situation is 
driven by Congressional requirements. 

With the 35 percent design complete, the project goes into the President's 
budget submission to the Congress for the DoD. It must then go through hearings 
within a number of subcommittees and committees within both houses of the Congress. 
If the project survives as a part of the Congressional budget process and is passed into 
law, it becomes legal for the Navy to enter into a contract to build the project. 

CONTRACTING 

There are several documents that implement Federal laws relating to 
construction within NAVFAC. These documents are part of a hierarchy, with the first 
having greatest power and the most generality. The lead document is the Federal 
Acquisition Regulations (FAR). These regulations govern all Federal procurement. 
The Department of Defense has a supplement to the FAR that publishes specific 
regulations pertaining to the DoD (DFARS). The Navy's supplement, the Navy 
Acquisition Procedures Supplement (NAPS), covers Navy-specific requirements. 
NAVFAC has its own Contracting Manual (P-68) which contains specific regulations 
applying to NAVFAC procurement. The standard method within NAVFAC and within 
the DoD for contracting for both A/E services and construction is the fixed-price 



contract. 
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A/E contracting is governed by the Brooks Act, which requires selection based 
on qualification with a contract price to be negotiated after selection is made. Price does 
not enter into consideration during the evaluation of a prospective A/E's qualifications. 
Generally, a change to the A/E contract will be negotiated at the start of construction for 
the designer to provide services such as submittal review during the construction 
period. The A/E is available as a resource for the ROICC in any questions regarding 
the design. A significant problem occurs with this contracting method during 
construction when changed conditions are encountered in the field or when the 
customer has a mission change requiring modification of the facility. Because of 
contracting regulations, a modification to the A/E contract must be negotiated before 
beginning design work required by a changed condition. This problem can result in 
delays in the field while the EFD negotiates the contract change with the A/E. 
Alternative contracting strategies, such as cost-plus design contracts, are allowed by 
the regulations, but their use is limited. 

Construction contracts within NAVFAC are almost exclusively competitively- 
bid, fixed-price contracts. Any contractor with experience and sufficient financial 
strength may bid on NAVFAC work. There are certain special programs, such as the 
Small Business Administration Section 8(a) Program, which exclude some bidders. 
These programs are in place to encourage success in the targeted businesses. Most 
MILCON projects have a general contractor who coordinates construction of the 
facility. Since 1988, most MILCON projects are bid on an unrestricted basis. 
Previously, many large projects had been "set-aside" for small business 
accomplishment. 
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Administration of the construction contract after award is the responsibility of 
the ROICC. The ROICC's Project Manager is the contractor's primary contact with the 
Navy for the day-to-day construction business. The Contracting Officer is the official 
government representative to the contractor. He is the person who has the authority to 
obligate the government for any additional money or time that may arise out of a 
changed condition. This individual may be either a Civil Engineer Corps Officer or a 
Contract Specialist (1 102 Series Civil Service employee). 

CONSTRUCTION CHANGE PROCESSING 

A formal procedure exists to handle changes on all NAVE AC construction 
contracts. The basic procedure is the same regardless of the amount of the change. 
Only the amount and detail of the documentation is different. What are called "change 
orders" in construction jargon are "Contract Modifications" to the government. A 
contract modification begins with a problem that is not covered within the contract. A 
letter from the contractor or a field observation starts the process. The Project Manager 
must do a quick estimate for the changed work to determine if there will be additional 
cost. At that point, he then makes a funding request through his Construction Area 
Manager in the Construction Division (Code 05) of the EFD. The Area Manager is a 
Civil Service engineer who is the central point of contact at the EFD for the ROICC. If 
the change is complex, the Project Manager also contacts the A/E to get design 
revisions for inclusion in the Request for Proposal (REP) to the contractor. 

After the contractor receives the REP, he has a given amount of time to forward 
his proposal to the ROICC's Project Manager. Once the proposal is received, the 
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Project Manager and Contract Specialist, as a team, develop the Government Pre- 
negotiation Position for review and approval by the Contracting Officer. After the 
Contracting Officer’s approval, they may then conduct negotiations with the contractor 
to determine a fair and reasonable price in money and time for the contract modification. 
Following completion of negotiations and the Contracting Officer's approval of the 
negotiated amount, a formal contract modification is signed by the Contracting Officer 
and the contractor. Field work may start after the modification is signed. 

SOUTHERN DIVISION CONSTRUCTION MANAGEMENT SYSTEM 

The Construction Management System (CMS) is the management information 
system used by the top executives within SOUTHDIV to track progress on all 
construction projects. An important component of the system, which has particular 
bearing on this research, is the modification tracking system. Each contract 
modification has a reason code that gives the executive a general explanation of why the 
change occurred. There are many category codes available. 

Two of the codes are particularly associated with errors or omissions in the 
project design. "EROM” stands for "Error or Omission." This category is used if A/E 
liability for the change is being investigated or if the A/E has agreed to pay for part of 
the change work. "DSGN" stands for "Design Deficiency." It is used when there is a 
designer mistake or a design omission for which the A/E has been found not 
responsible. The category "UNFO" covers unforeseen conditions. 

One problem that occurs with the coding system is the determination of A/E 
liability for a change. Both EROM and DSGN codes require a determination of A/E 
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liability for the changed condition. That is an extra step in the process of obtaining 
funds to execute a contract modification. In some cases, it is not obvious whether the 
A/E is liable. When that occurs, there can be delay in receipt of funds at the field level. 
On an item that has impact on the critical-path of a project, that delay can create 
entitlement to compensation for extended overhead expenses for the construction 
contractor. Delay costs are very expensive and they do not buy any additional 
construction. 

Assignment of the reason code is normally made by the Construction Area 
Manager in Code 05 at SOUTHDIV. The code must be included in a request to Project 
Management (09A2) for funds to begin the modification process. The Project Manager 
in the field office may have input into the reason code, but the Area Manager makes the 
final determination. Sometimes, to speed up release of funds to the field, the Area 
Manager codes a change that perhaps could fall in the EROM or DSGN categories as 
UNFO. Coding the change UNFO eliminates the need for analysis to find out if the 
A/E is liable for the increased costs. This could cause the statistics reported by change 
code to be inaccurate. 



CHAPTER 3 



DESIGN INPUTS 

Design is a very complex process that takes the owner's ideas and desires and 
transforms them into drawings and specifications that are used by a constructor to build 
the desired facility. Many decisions take place in this process that will affect the 
completed facility. 

The Construction Industry Institute (CII) established the Design Task Force in 
the spring of 1984 to study the management of design. They organized their study into 
three general areas: inputs to the design process, the design process itself, and outputs 
of the design process. ^ In Publication 8-2, Input Variables Impacting Design 
Effectiveness , the task force identified the ten input variables having the greatest impact 
on design effectiveness. They are:^ 

• Scope Definition 

• Owner Profile and Participation 

• Project Objectives and Priorities 

• Pre-Project Planning 

• Basic Design Data 

» Designer Qualification and Selection 

• Project Manager Qualifications 

• Construction Input 

^Construction Industry Institute Design Task Force, Evaluation of Design Effectiveness. 
Publication 8-1 (Austin, TX: Construction Industry Institute. 1986), p. 1. 

^Construction Industry Institute Design Task Force, Innut Variables Imnactine Design 
Effectiveness . Publication 8-2 (Austin, TX: Construction Industry Institute, 1987), p. v. 

12 



13 



• Type of Contract 

• Equipment Sources 

NAVFAC DEFINITIONS OF THE INPUT VARIABLES 

The CII definitions of each of these terms were oriented toward private sector 
work. In his dissertation, Broaddus defined each of these variables in Navy 
terminology. The discussion that follows center on those definitions. 

SCOPE DEFINITION 

"Scope Definition" in the NAVFAC sense is the process of filling in project data 
listed on the Department of Defense Project Documentation Form (DD-1391). 
Theoretically, if the form is completed properly, the designer will have the information 
required to design the project efficiently and with minimum changes. Information 
required to complete the form includes size, functional requirements, budget, project 
requirements, etc. Unfortunately, this process does not always work as intended. 
Because of recent problems with this process, NAVFAC instituted a new program in 
1988 to improve scope definition. This program is "Certified Ready for Design." 
This process is to take place before detailed design begins. It involves a comprehensive 
review of environmental impacts of a facility and numerous special features and 
requirements typical for the particular type of facility being considered. It includes any 
special construction considerations, long-lead item procurement, real estate and land 
acquisition issues, and seismic requirements. The scope needs to emphasize, or 
perhaps even prioritize, features required in case tradeoffs are required as the design 
develops. Ultimately, scope definition is a description of what the facility must do for 
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the user. It must be clear enough so the A/E can translate it into meaningful plans and 
specifications.'^ 

OWNER PROFILE AND PARTICIPATION 

Broaddus defmed this input item as the "Participation of the facility user, major 
claimant and local activity in providing timely and accurate project requirements and in 
effective and consistent decision-making throughout the planning and design process."^ 
This input variable is difficult. In the traditional sense, NAVFAC is the owner's 
representative, but the user, major claimant and local activity fulfill the role envisioned 
by the CII definition of this variable. They are responsible for executing their mission 
once the facility is completed. Changes in their mission requirements drive changes in 
the design, and their input is critical to ensuring the facility will satisfy their needs. 
NAVFAC's role is covered later under Project Manager Qualifications. 

PRE-PROJECT PLANNING 

The closest approximation of this input variable in NAVFAC is the project 
acquisition strategy. Broaddus defined it as the "Adequacy of the acquisition strategy 
for completing the project in an efficient manner. . . The acquisition strategy is the 
plan for construction and procurement that is determined at the EFD in the early stages 
of a project. 



'^Broaddus, pp. 66-67. 
^ Ihid .. p. 68. 

^Ibid.. p. 68. 
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PROJECT OBJECTIVES AND PRIORITIES 

This input variable covers the project objectives that are most important to the 
user and major claimant. They are defined in terms of functional requirements, 
importance of aesthetics, project execution schedule, expandability, level of 
technology, initial operating date, etc.^ There is no special effort to define specific 
project objectives in writing. The "Certified Ready for Design" program deals with 
objectives, but they are not specifically addressed. Usually, the main project objective 
for the user is to have the facility available when needed for some critical mission 
requirement. 

BASIC DESIGN DATA 

The adequacy of NAVFAC Guide Specifications, design manuals, and standard 
designs for a particular type facility are a part of this input variable. Also included is 
the adequacy of any user supplied basic data for unique or specialized-type facilities 
(i.e., process requirements, environmental controls, etc.), and the extent to which basic 
design data was beneficial and not a constraint to the design process.^ NAVFAC's 
standard publications and guide specifications provide guidance to the A/E in the 
preparation of the design for a project. In some cases, the basic design data are out-of- 
date and prevent the A/E from specifying a newer, more efficient process or product in 
the facility. 



^Ihid.. p. 69. 
^Ibid.. p. 70. 
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SELECTION AND QUALIFICATION OF DESIGNER 

The selection of designers for NAVFAC projects falls within the Brooks Act. 
This law specifies that designers must be selected on the basis of technical merit and not 
cost considerations. A/E’s are selected on technical performance, management ability, 
experience with similar projects, capacity to handle the work, etc. Only after selection 
is made is the pricing of services negotiated. This process generally produces a quality 
selection, but, because not all firms are alike, there is significant variance in 
performance by A/E’s on NAVFAC projects.^ 

PROJECT MANAGER QUALIFICATION 

Broaddus defined this input variable as the impact of the NAVFAC Engineering 
Field Division people assigned to the project, their qualifications, the consistency of 
personnel assigned (turnover), and their contribution to the project. There is a 
Project Manager in Code 09A2 who is theoretically responsible for the project from 
conception to completion. However, during various phases of the process, certain 
other people exercise what would normally be thought of as project manager 
responsibilities. During the planning phase (conception to design authorization), the 
Head Planner is the lead. At design authorization, the project is turned over to the 
Project Manager, who takes it to completion. During the design phase, there is an 
Engineer-in-Charge in Code 04 (Design Division) who keeps up with the designer's 
progress and is the A/E's primary contact at the EFD. During Construction, the 
Construction Area Manager in Code 05 (Construction Division) is involved with 



9lhid.. pp. 70-71. 
lOlbjd., p. 71. 
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coordination at the EFD for the project. The final player is the ROICC. He is the 
Navy's representative to the construction contractor. He is responsible for the project 
execution in the field from award through completion and startup. Any evaluation of 
input variables is effected by all these players, so the definition includes the entire EFD 
project team. 

CONSTRUCTION INPUT 

This input variable is the thoroughness of construction input provided during 
the design phase from contractors, Construction Division personnel, and ROICC 
personnel concerning the availability of labor and materials, appropriate construction 
methods, sequencing of work, practical advice on field conditions, etc.^^ There is 
generally little construction input during the design phase of a project and the input that 
does occur comes very late in the process. A constructability program as advocated by 
CII is not in place. The Construction Area Manager and ROICC personnel review the 
design at various stages of completion and make written comments. The only review 
by contractors occurs after the project is out for bids and the contractors ask questions 
to clarify the designer's intent on a specific item or section of the project. 

TYPE OF CONTRACT 

Broaddus defined this variable as the effectiveness of the A/E contracting 
process (i.e., length of the contracting process and its impact on project schedule, 
methods for handling scope changes. Government design review procedures, level of 
responsibility and authority given to the designer, etc.j.i^ The type of contract used for 



1 llbiil., p. 72. 
12li>id., p. 72. 
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A/E services within NAVFAC is almost exclusively the fixed-price contract. This can 
have significant impact on project schedule if a change to the contract is required. 
Regulations require that a negotiated contract change be executed before any work by 
the A/E. This involves Contracts Department actions that sometimes include significant 
processing time. Sometimes this problem is overcome by the A/E proceeding with the 
work at his own risk, but the Government ends up paying for the A/E's risk. The trend 
is for more control of the process by Contract Specialists to ensure both Government 
and A/E compliance with current procurement regulations. New procedures may have 
impact on this input variable, but all 55 projects considered in this research were 
designed before the procedural changes took place. 

EQUIPMENT SOURCES 

This variable is the completeness, timeliness and firmness of vendor and 
Government furnished equipment data. The impact of this variable will depend on the 
type of project being considered. Generally, most commercial-type facilities do not 
have Government furnished equipment. All equipment is specified as part of the 
construction contract and the contractor handles all procurement. In some industrial 
facilities, the Government provides equipment and the accuracy of forecasts for this 
equipment can have an impact on design effectiveness. The Government must be able 
to give the A/E a manufacturer for a particular piece of equipment and the manufacturer 
must provide data regarding size, power requirements, etc., for the designer to 
complete the design. If this information is not accurate or is late, it could have an 
impact on the project schedule. Also, if equipment is a part of the construction 
contract, the specifications may not favor any particular manufacturer over another. 
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The specification must allow competition between equipment sources. This can 
sometimes make it difficult for the A/E to get the equipment he desired when he 
designed the facility. This is a problem area for NAVFAC.^^ 

DESIGN INPUT INDEX 

To tie all these input variables together and provide a meaningful research tool, 
Broaddus developed a questionnaire to quantify the impact of each of the input 
variables for the projects he studied. He used the objectives matrix technique and, in 
the process, developed a measure of design input called the Design Input Index. This 
index varied from 100 (very poor) to 1000 (superior). This research studied the same 
55 projects as Broaddus and used his Design Input Index values for comparison with 
change information on those same projects. The objectives matrix technique is 
explained in great detail in CII Source Document 22, Objectives Matrix Values for 
Evaluation of Design Effectiveness , by John O. Stull, Jr. and Richard L. Tucker. 



^3lhid.. p. 73. 
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RESEARCH METHODOLOGY 

DATA GATHERING 

As mentioned in the previous chapter, data for Design Input Index came from 
Broaddus. Input index and general project information were placed in a spreadsheet in 
Microsoft Excel for use as a database in the analysis. Data for changes on those same 
projects came from Southern Division. This information was available directly from 
the Construction Management System database. It came in the form of a project 
printout with all changes listed by category, cost and description. Change category and 
cost figures were entered in the database for use in the analysis. Primary emphasis was 
on the number of changes in the EROM, DSGN and UNFO categories. EROM and 
DSGN were chosen because they reflect problems with the project design. Since 
Design Input Index is a measure of the quality of the design, the categories tied to the 
design should have some relationship with the input index. UNFO was chosen 
because of the assumption that a more well defined design should have fewer 
unforeseen conditions. Total changes, the total cost of changes on each project, and 
costs associated with each change category were also considered important because 
they gave some indication of the overall change behavior of the projects and tied an 
important asset, money, to the analysis. There was no further investigation into the 
specific work accomplished in each contract modification, because the emphasis of this 
research was on overall change behavior by change category. 
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ANALYSIS METHODS 

The primary analysis methods were correlation analyses using least-squares 
principles. Several computer software packages were used for various parts of the 
analysis. StatWorks™ by Cricket Software, Inc. was the first package employed and 
was used for the statistical analysis. It provided information regarding the Coefficient 
of Determination, R^, for each of the comparisons. To provide a check on 
StatWorks™ and to confirm the results, DeltaGraph™ by Deltapoint, Inc. was used. 
DeltaGraph™ also was used for the graphics in this report. Microsoft Excel was used 
for the database. 

Statistical comparisons were made using the "curve fit" or "regression" options 
on both programs. The independent variable in the analysis was the Design Input 
Index for all projects and the dependent variable was number of changes or change cost 
for a specific category for all projects. The input index and change data for a specific 
project were matched together to create a data point in a two-dimensional plot. The 
initial focus of the study was to develop an easy to use linear model to relate design 
input index to changes on projects. Because of this, the first comparison done was a 
linear regression. Before any computer analysis could begin, the cutoff level for the 
coefficient of determination, R^, had to be defined. From Young's Statistical 
Treatment of Experimental Data , there was a 10 percent probability that a R^ value of 
0.23 could be exceeded by random chance with 55 observations.^^ Since the sample 
for this research was 55 projects, a R^ level of 0.23 could occur even if the variables 
being compared had no correlation. The value of the coefficient of determination for a 

^‘^Hugh D. Young, Stati.stical Treatment of Experimental Data (New York; McGraw-Hill 
Book Company, Inc., 1962), p. 16. 
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mcxlel to show a minimal relationship in this research was set at 0.40. This cutoff point 
was selected so low because the data come from different projects with different 
management and ideas regarding the importance of the category coding system. The 
variables had to be related beyond that which could be expected because of random 
matching. 

To do the analysis, StatWorks™ used the principles of least-squares. The 
program took the data sets for each comparison and fit the "best line" to the data. The 
assumptions made in the program follow those of standard least-squares theory. -phe 
theory is explained in great detail in Draper and Smith's book. Applied Regression 
Analvsis .^^ The concept most important to this research was that the value showed 
whether the variables being compared had a reasonably linear relationship. If two 
variables could have a perfect linear relationship, the value for the comparison of 
those two variables would be 1.0. In practice, and with data from different sources 
subject to personal biases, a value of 1.0 would be impossible. Based on the previous 
paragraph, a R^ value of 0.23 could occur with 55 random observations, so if a value 
of 0.23 or below were to occur, that comparison would not have a linear relationship. 

StatWorks™ can also do polynomial regressions. The program used least- 
squares principles for the polynomial analysis. Polynomial regressions of various 
degrees (2nd, 3rd and 4th) were also done on each data set. The program gave the 
same basic information about each polynomial comparison as it did for the linear 



*^James Rafferty, Richard Norling, Robert Tamaru, Charles McMath and Dave Morganstein, 
StatWorks™ (Philadelphia, PA: Cricket Software, 1985), pp. 53-59. 

R. Draper and H. Smith, Annlied Regression Analysis (New York: John Wiley & 
Sons, 1981), pp. 8-40. 
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regression analysis. The key statistic was R^. The same cutoff level of 0.40 for a 
good model in linear regression was used in the polynomial analysis. 

The final comparison done as part of the analysis was to see if a logarithmic 
curve would "fit" the data. To do this in StatWorks™, the data for the dependent 
variable had to be transformed to logarithm by the program. Once this step was 
complete, a linear regression could be done to define the curve. The results were the 
same as the linear regression done earlier. The coefficient of determination, R^, was 
the important result. The value for determining whether the model was satisfactory was 
the same as for the linear regression. 



CHAPTER 5 



PRESENTATION OF DATA 

The projects studied in this research were all within Southern Division of 
NAVFAC. They all had a dollar value over $2 million and were all finished or 
scheduled to finish within a one-year window around July 1990. The cutoff date for all 
information included in the analysis was July 1990. A detailed listing of the data in 
spreadsheet format is in the Appendix. 

DESIGN INPUT INDEX 

Design Input Index varied from a low value (worst) of 158 for the Electrical 
System Distribution Improvements at the Naval Shipyard, Charleston, South Carolina 
to a high value (best) of 800 for the Ship Berthing Improvements at Naval Station, 
Mayport, Florida. Figure 5-1 shows the variation in Design Input Index for the 55 
projects. Each bar is a project. The average Design Input Index was 572. 

CHANGE DATA 

The total number of changes had a low value of 4 for three different projects. 
Those were the Drydock Pumphouse at Naval Shipyard, Charleston, South Carolina; 
the Reserve Training Center in Greenville, South Carolina; and the Magazines - Phase 1 
in Ingleside, Texas. Figure 5-2 shows the variation in Total Changes. 

The variation in the total cost of changes is shown in Figure 5-3. The low 
dollar-value of changes occurred on the Missile Magazines at Charleston Naval 
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Weapons Station. This project had a net credit back to the Government for changes of 
over $14,000. The high project was the Electrical System Distribution Improvements 
at Naval Shipyard, Charleston, South Carolina with a total change cost of $2.1 million 
on a contract originally awarded at $4.9 million. This project had a current value in 
July 1990 of $7 million. 
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PROJECTS 



FIGURE 5-1: DESIGN INPUT INDEX 

For the purposes of this research, changes in the EROM and DSGN categories 
were combined in the analysis. This was done because both categories concern design 
errors and the number of EROM changes was so small that no meaningful analysis 
could be done on that category alone. Figure 5-4 show the total number of EROM and 
DSGN changes. There were two projects that had no EROM or DSGN changes. They 
were the Headquarters Support and Telephone Building at Naval Station, Ingleside, 
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Texas and the T-45 Squadron Maintenance Facility at Naval Air Station, Kingsville, 
Texas. The project with the highest number of FROM and DSGN changes was the 
Aircraft Structural Repair Facility at the Naval Aviation Depot, Pensacola, Florida with 
25. 
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FIGURE 5-2: TOTAL CHANGES 





EROM + DSGN CHANGES (#) TOTAL CHANGE COST ($) 
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FIGURE 5-3: TOTAL CHANGE COST 
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FIGURE 5-4: NUMBER OF EROM+DSGN CHANGES 
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The cost of EROM+DSGN changes is shown in Figure 5-5. The two projects 
with no FROM or DSGN changes also had no cost associated with those two 
categories. They were not the low projects, however. The Ship Support Complex at 
Naval Station, Ingleside, Texas had a FROM -i- DSGN cost that was a credit to the 
Government of $42,282. The high project was the Modifications to the Bachelor 
Enlisted Quarters at Marine Corps Air Station, Beaufort, South Carolina. 




FIGURE 5-5: EROM + DSGN CHANGE COST 

The total number of UNFO changes is shown in Figure 5-6. The project with 
the most UNFO changes was the Electrical System Distribution Improvements at the 
Naval Shipyard, Charleston, South Carolina. There were five projects that had no 
UNFO changes. They were the Missile Magazines at the Charleston Naval Weapons 
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Station; the Shore Intermediate Maintenance Activity at Naval Station, Ingleside, 
Texas; the Magazines - Phase 1 at Naval Station, Ingleside, Texas; the Storage Facility 
at the Naval Supply Center at Jacksonville, Florida; and the Aircraft Maintenance 
Hangar at Naval Air Station, Memphis, Tennessee. 
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PROJECTS 



FIGURE 5-6: NUMBER OF UNFO CHANGES 



The five projects previously listed with no UNFO changes also had no cost 
associated with that category, but they were not the low cost for this change code. The 
Underwater Research and Development Facility at Naval Coastal Systems Laboratory' 
in Panama City, Florida was lowest with a credit to the Government of $67,05 1 . The 
Electrical System Distribution Repair at the Naval Shipyard in Charleston, South 
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Carolina had the highest UNFO cost ($1.4 million). The UNFO change cost is shown 
in Figure 5-7. 




FIGURE 5-7: UNFO CHANGE COST 

These figures show a graphical representation of the variables considered in the 
analysis. The complete spreadsheet with project names and the specific values for the 
different variables is included in the Appendix. 




CHAPTER 6 



ANALYSIS OF DATA 

TOTAL NUMBER OF CHANGES 

The data for Total Changes versus Input Index is shown in Figure 6-1. The 
scatter of the data prevented construction of a statistically significant model using 
regression techniques that could predict behavior of total number of changes with 
known Design Input Index. Linear, polynomial, and logarithmic analyses were done 
on the data. The coefficient of determination (R^) for all fits was below 0.20. The data 
does have some value, however. Except for three projects, the total number of changes 
appeared to focus toward a lower number as input index increased. At lower input 
index, there was more scatter. This is an imponant result, because it tells the Navy 
that, with a low input index, the change results will be less known than if the input 
index is higher. With the focussing of the data, there should be fewer changes on the 
project. This focussing could be anticipated knowing the elements that make up the 
Design Input Index. With better project definition, experienced construction people 
would expect there to be fewer changes on the job. 

TOTAL CHANGE COST 

Figure 6-2 shows the data plot for Total Change Cost versus Design Input 
Index. Analysis of these data gave a very flat reading. The correlation for a linear fit 
was less than 0.09. The flatness of the data indicates that there is very little correlation 
between the cost of the total number of changes and the Design Input Index. While 
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FIGURE 6-1: TOTAL OF CHANGES VS. DESIGN INPUT INDEX 




FIGURE 6-2: TOTAL CHANGE COST VS. DESIGN INPUT INDEX 
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there is more scatter in the plot at low input index, it is not substantial. Most of the data 
points are grouped below $500,000, regardless of their input index. 

NUMBER OF EROM + DSGN CHANGES 

The number of EROM + DSGN changes is shown plotted against Design Input 
Index in Figure 6-3. In this case, the data are scattered all over the graph. Coefficients 
of determination for this comparison were very low for all categories, so no statistical 
model could be developed using regression techniques. The same observation about 
the data for total number of changes holds true in this case also. The data has an 




FIGURE 6-3: NUMBER OF EROM -h DSGN CHANGES VS. DESIGN 
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apparent focus toward the low end of the cost scale, with the exception of three 
projects, when input index is increased. This is very important because it shows that it 
is possible that, if input index is increased, there will be fewer design-related changes 
during construction. Certainly there is more scatter, and therefore less certainty, at or 
below an input index of 500. Most of the focussing in this area occurs after the 500 
level. 

EROM DSGN CHANGE COST 

Analysis on the cost of EROM -i- DSGN changes versus the Design Input Index 
did not yield any statistically significant models. All coefficients of determination were 
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FIGURE 6-4: EROM -h DSGN CHANGE COST VS. DESIGN INPUT 



INDEX 



35 



below 0.08 for linear and polynomial approaches. The data in this area is mostly 
grouped between zero and $150,000. There are some projects outside those bounds, 
but the majority are within them, regardless of input index. There is no apparent impact 
of higher input index on the cost of EROM + DSGN changes. Figure 6-4 shows the 
plot of EROM -f DSGN cost versus input index. 

NUMBER OF UNFO CHANGES 

Figure 6-5 shows the number of UNFO changes against input index. Again, 
the statistical analyses did not give any significant results. Coefficients of 




FIGURE 6-5: NUMBER OF UNFO CHANGES VS. DESIGN INPUT 
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determination were below 0.10 for all models. The same observations regarding 
focussing of data discussed before hold true for this category. The data tends to focus 
to the low end of the number of changes axis as the input index increases. This result 
could be anticipated knowing the make-up of the Design Input Index. As the project is 
more well defined, there should be fewer unforeseen problems. 

COST OF UNFO CHANGES 

The plot of UNFO change cost against input index turned out to be fairly flat 
(see Figure 6-6). Coefficients of determination for this category were all below 0.20, 
so there were no statistically significant models for the relationship between the two 
variables. The data tend to focus toward zero as input index increased. This result is a 
symptom of the items that make up the Design Input Index. As scope is defined better, 
as the owner/customer participates properly in the process, as objectives are defined 
clearly, etc., the project can be expected to have less unforeseen costs, because there 
are fewer items that can be called unforeseen. The planning process creates a 
mechanism where the project becomes more clear and the design can reflect actual 
conditions and technologies that the contractor can translate into a finished structure. 

SUMMARY 

In looking at total changes, the selected change categories and costs associated 
with them, no significant statistical models based on regression analysis could be 
developed from the data. The trends in the data can provide a guide to the benefits of 
increasing the value of the Design Input Index to projects. Though a mathematical 
model was not developed out of this analysis, the results are nonetheless valuable. 



UNFO CHANGE COST ($) 
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FIGURE 6-6: COST OF UNFO CHANGES VS. DESIGN INPUT 

INDEX 



CHAPTER 7 



CONCLUSIONS 

1 . The analysis did not yield a statistical model, based on regression analysis, for 
change behavior on NAVFAC projects based on the Design Input Index as formulated 
by Broaddus. 

2. Design Input Index is a good relative measure of change behavior for projects 
within the Southern Division area. It will not give an exact statistical answer for the 
number of changes in each category to expect on a given project, but it is a good 
predictor of the order of magnitude of expected changes. The benefit to increasing the 
input index is to have more certainty about the order of magnitude of changes. At low 
input index, the change behavior could be anywhere from low to extremely high. 

3. The trend shown in the analysis for total changes, number of EROM and 
DSGN changes, number of UNFO changes and the cost of UNFO changes of 
focussing toward low values as input index increases shows that knowing the Design 
Input Index is important. Knowing that value, the senior leadership in the organization 
can have a feel for the health of a given project. If input index is high, they are 
probably in good shape. If it is low, more effort needs to be made to define the items 
which make up the index. 

4. Results of this analysis are useful for the ROICC in the field because they 
provide a guide for how a given project may turn out. Knowing Design Input Index, 
the ROICC or his senior representative can assign projects to Project Managers within 



38 



i.;; 






39 



the office. Changes make up a majority of the Project Manager's work effort. If the 
ROICC knows the project may have a great many changes, he may want to assign that 
project to a more experienced individual. He can distribute the projects within the 
office to maximize the individual's experience and growth, but without sacrificing 
success of the project . 

5 . There is no apparent correlation between Design Input Index and the direct cost 
of changes for Total Changes and EROM + DSGN changes. Indirect costs to the 
government (preparation time for estimates, negotiations, etc.) are not currently 
quantifiable, but those items are a substantial portion of a ROICC Project Manager's 
job. Processing of contract modifications prevents the Project Manager from getting 
out into the field to observ'e and analyze their projects. The lack of direct correlation 
should not be taken as an indication that greater effort to improve the quality of design 
input would be wasted. 



CHAPTER 8 



RECOMMENDATIONS 

ACTION BASED ON THIS RESEARCH 

1. Because there is benefit in minimizing changes to projects in both time and 
money, NAVFAC should perform surveys to quantify Design Input Index for all 
projects in all phases of the design process. It should begin with the planning process 
and carry through until detailed design is completed. The design process milestone 
times currently in use would be ideal points to begin. Once Design Input Index is 
quantified, it should be used to identify problem projects early, so additional resources 
can be dedicated to make the project successful. 

2. The coding of changes needs to be standardized better in all offices for the 
Construction Management System to be of more use to executives in the organization in 
tracking changes. The bureaucratic process for both EROM and DSGN changes needs 
to be streamlined so that a ROICC in the field does not have to code a change as UNFO 
just to get funds quickly to avoid delay damages. The A/E liability question should be 
pursued, but not at the expense of the project schedule. The EFD should fund the 
change up front and seek funds from the A/E after the fact if they were liable for the 
change. 

3. There needs to be more training to familiarize people in the field with the 
different change categories and the importance of proper coding in the overall 
Construction Management System. 



40 



41 



FUTURE RESEARCH 

1. It would probably be beneficial to look into the issue of improper coding of 
changes to quantify the practice. Once the leaders of the organization know the scope 
of the problem, it can be corrected. These statistics are valuable tool to track projects, 
but if the reporting is inaccurate, those using them may make incorrect decisions. 

2. Investigation of project schedule delays and their relationship with Design Input 
Index would be another area which could follow-on to both this work and that of 
Broaddus. 

3. An investigation into where the design errors are made that are categorized as 
EROM or DSGN could tell NAVFAC where they need to concentrate more effort 
during the planning and design stages of projects. 



CHAPTER 9 
APPENDIX 
Project Data 
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LOCATION CONTRACT PROJECT CURRENT INITIAL AWARD INPUT INDEX # EROM EROM COST 

NO VALUE 
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ORLANDO NTCl^ 84-0999 WEAPONS TRAINING FACH.ITY $2,501,565 $2,483,000 855 0 $0.00 

PANAMA CITY NCSC 85-0450 UNDERWATER R & D FACILITY $3,998,543 $3,974,780 550 0 $0.00 
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CHAPTER 10 



GLOSSARY OF TERMS 

CERTIFIED READY FOR DESIGN: A NAVFAC program to improve the 
quality of scope definition on Navy MILCON projects. 

DESIGN INPUT INDEX: A measure of design effectiveness developed by James 
A. Broaddus in his doctoral dissertation which has a value from 100 (poor) to 1000 
(superior). 

DSGN: The NAVFAC contract modification reason code denoting a "Design 

Deficiency." This category is used when there is a designer mistake or a design 
omission for which the A/E has been found not responsible. 

ENGINEERING FIELD DIVISION (EFD): A branch of NAVFAC which is 
responsible for execution of the facilities program within a defined geographic area. 
The EFD is responsible for planning and execution of the Military Construction 
Program for their area. 

EROM: The NAVFAC contract modification reason code denoting an "error or 
omission" in the project design. This category is used if A/E liability for the change is 
being investigated or if the A/E has agreed to pay for part of the change work. 

MILITARY CONSTRUCTION (MILCON) PROGRAM: The Navy shore 
facilities capital improvement program. All new construction projects with dollar value 
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over $200,000 are included in this program which is authorized annually by the 
Congress as a part of the Department of Defense budget. 

NAVAL FACILITIES ENGINEERING COMMAND (NAVFAC): The 
engineering organization within the United States Navy responsible for all Navy shore 
facilities. 

RESIDENT OFFICER IN CHARGE OF CONSTRUCTION (ROICC): 
The field office of the EFD responsible for actual field construction of new projects. 
They administer the construction contract after it is awarded. 

SOUTHERN DIVISION (SOUTHDIV): The EFD responsible for the 

geographical region in Southern United States from South Carolina to New Mexico. 

UNFO: The NAVFAC contract modification reason code denoting unforeseen 
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